A paper sheet recognition apparatus that acquires an image and a spectrum of optical characteristics of a paper sheet, and recognizes a type and authenticity of the paper sheet based on the acquired information is known in the art. For example, Patent Document 1 discloses an apparatus that irradiates a paper sheet with a light of a predetermined wavelength, and recognizes authenticity of the paper sheet based on a feature appearing in optical spectrums of fluorescence and phosphorescence that are excited on the paper sheet. Moreover, Patent Document 2 discloses an apparatus in which a plurality of light sources capable of irradiating a paper sheet with lights varying in the angles of irradiation are arranged, and presence or absence of a hologram on the paper sheet is determined based on a difference in reflection spectrums obtained by emitting lights from these light sources. In this apparatus, authenticity of the paper sheet is recognized from the presence or absence of the hologram on the paper sheet.